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WWC identified 69 
studies of 32 programs 
and practices 




8 studies 
met evidence 
standards with 
reservations 




English language learners are students with a primary language 
other than English who have a limited range of speaking, reading, 
writing, and listening skills in English. English language learners 
also include students identified and determined by their school 
as having limited English proficiency and a language other than 
English spoken in the home. 

This review focuses on programs to improve the English 
language literacy and/or academic achievement of elementary 
school students who are English language learners. For a 
program to be included, the majority of instruction has to be 
conducted in English (approximately 80%) but teachers might 
occasionally provide some native language support. The review 
focuses on programs for English language learners in grades 
K-6. It also includes studies in which students may no longer be 
considered to have limited English proficiency by their school but 
still have limited English language skills. 

We looked at 73 studies of 32 programs that qualified for our 
review. Of these, 16 studies of 12 programs met our evidence 
standards— 8 without reservations, and 8 with reservations. 



Twenty programs reviewed had no studies meeting WWC 
evidence screens, three of which were eligible for inclusion 
in this topic review but did not have studies. Additionally, 20 
studies could not be categorized by an intervention and are not 
cited in the report.'' (The initial identification of eligible programs, 
practices, and studies ended in March 2006). 

The review focused on three outcome domains: reading 
achievement, mathematics achievement, and English language 
development. 

In looking at the three outcome domains for the 12 interven- 
tions, the following produced potentially positive effects on 
reading achievement and English language development: 

• Bilingual Cooperative Integrated Reading and Composition 

• Instructional Conversations and Literature Logs 

• Vocabulary Improvement Program 

Eight other programs and practices had potentially positive 
effects in one domain. One program had no discernible effects in 
two domains (see the WWC Intervention Rating Scheme) . 



This review summarizes the first 
wave of intervention reports 
produced in 2005-06. 

www.whatworks.ed.aov 1. Four additional studies use single-case designs and are still under review. The WWC is currently developing standards for the review of single-case studies. 
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Intervention ratings for English language learners 

Each English language learning program that had at least one study meeting in the intervention and comparison conditions, and the consistency in findings 

WWC standards with or without reservations received a rating of effectiveness in across studies. 

at least one outcome domain: reading achievement, mathematics achievement. The research evidence can be rated as positive, potentially positive, mixed, no 

and English language development. The rating aims to characterize the evidence discernible effects, potentially negative, or negative. Table 1 shows the effective- 
in a given domain, taking into account the quality of the research design, the sta- ness ratings for the 12 programs and practices in the three domains (empty cells 

tistical significance of the findings, the size of the difference between participants indicate that no evidence was reported). 




Intervention name 



Reading achievement Engiish language development Mathematics achievement 



Arthur (http://pbskids. org/arthur/index. html) 






Bilinaual Coooerative Inteorated Readino and 
Comoosition (httD://www.successforall.ora) 




if 




Enhanced Proactive Readino (httD://www.texasreadina.ora/utcrla) 


(if 


Q 




Fast ForWord Lanouaoe (httD://www.scileam.com/Drovidersearch) 


o 


if 




Instructional Conversations and Literature Loos 

(http://crede. berkeley. edu/research/crede/researcb/Haa/1. 5pubs. html) 


Of 


if 




Peer Tutorino and Resoonse Grouos (no website available} 








Peer-Assisted Learnino Strateoies (httD://kc.vanderbilt.edu/Dals) 


if 






Read Naturallv (httD://www.readnaturallv.com} 


Q 






Read Well (httD://store.cambiumleamina.com/ProaramPaae.asox?DarentlD 
=019005451&functionlD=009000008&plD=Pead%Well&site=sw) 


if 






Readino Mastery (httD://www.sraonline.com) 


if 






Success for All (httD://www.successforall.com) 


if 






Vocabulary Imorovement Prooram 

(http://www.brookespublishing.com/store/books/lively-6342/index.htm) 


if 


if 





Note: The Extent of Evidence Categorization, developed to tell readers how much evidence was available to determine the intervention rating for each study, is in Appendix A1 . Three programs had research findings in two domains, and eight 
programs had research findings in one domain. No programs had research findings in fhe mathemafics achievement domain. WWC intervention reports describe each program and provide information on the students, the cost, and 
the scope of use. To view the intervention reports, please click on the program name or go to www.whatworks.ed.aov . Following each program name is the developer's website address. The evaluated research addresses some but 
not all grade levels targeted by these Interventions. Grade levels are related to student age and may affect outcomes. For a comparison of targeted grade levels and grade levels in the studies reviewed by the WWC, see Appendix A2. 



Key 




o 



with no overriding 
contrary evidence 



negative effect 



negative effects: 
evidence of a 



Potentially 




Negative effects: 
strong evidence of 
a negative effect 
with no overriding 
contrary evidence 
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Average improvement indices for each domain 

The WWC computes an improvement index for each individual finding. In addition, 
within each outcome domain, the WWC computes an average improvement index 
for each study as well as an average improvement index across studies (see 
Technical Details of WWC-Conducted Computations) . 

The improvement index represents the difference between the percentile rank 
of the average student in the intervention condition and the percentile rank of the 
average student in the comparison condition. The improvement index can take on 
values between -50 and +50, with positive numbers denoting results favorable to 
the intervention group. Unlike the rating of effectiveness, which is based on four 
factors, the improvement index is based only on the size of the difference between 
the intervention and the comparison conditions. 

Reading achievement 

The reading achievement domain consists of student literacy outcomes includ- 
ing phonemic awareness, phonics, reading fluency, vocabulary, and reading 
comprehension. The vocabulary measures included in the reading achievement 
domain consist only of measures that deal with the ability to define or use words 
that appear in text the student reads. We reviewed outcomes in this domain for 



Figure 1 Reading achievement: average improvement 
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10 English language learning programs, and the average improvement index 
ranged from 0 to +29 percentile points (figure 1). 

Mathematics achievement 

The mathematics achievement domain includes measures of content knowledge 
(number sense, properties, and operations; measurement; geometry and spatial 
sense; data analysis, statistics, and probability; and algebra and functions) as well 
as an understanding of mathematical concepts, procedures, and problem solving. 
Mathematics instruction involves the use of language, and thus we are interested 
in effective methods for teaching mathematics to students with limited proficiency 
with the English language. In this topic review, no studies that met WWC evidence 
standards with or without reservations addressed mathematics achievement. 

Engiish language development 

The English language development domain consists of oral or written proficiency 
outcomes including measures of listening comprehension, receptive vocabulary, 
grammar, syntax, and other linguistic features of the English language. The vocab- 
ulary associated with English language development relate to either expressive or 
receptive language in testing situations with no text to be read. We reviewed out- 
comes in this domain for eight English language learning programs and practices, 
and the average improvement index ranged from -1 to +31 (figure 2). 



Figure 2 English language development: average improvement 
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Table 2 Programs reviewed with no studies meeting WWC evidence screens 



Accelerated Reader (http://www.renleam.com/ar) 


Metacognitive Teaching Approaches (no website available) 


Augmenting Thinking Through Language Acquisition Skills (ATTLAS) (no website available) 


NEARStar (http://www.nearstar.org) 


Cognitive Academic Language Learning Approach (CALLA) (www.gwu.edu/~calla) 


On Our Way to English (http://rigby.harcourtachieve.com/en-US/rigby.htm) 


Curriculum-Based Instruction (no website available) 


Pre-Teaching Vocabulary (no website available) 


Effective Use of Time (no website available) 


Project MASTER (no website available) 


ESL in the Content Areas (no website available) 


Reading Recovery (http://www.readingrecovery.org) 


Front Row Phonics (no website available) 


Sheltered Instruction Observation Protocol (http://www.cal.org/siop) 


Instructional Practices (no website available) 


Story Structure Strategies/Story Structure Utilization (no website available) 


Into English (no website available) 


Structured Immersion (no website available) 


Learning Centers (no website available) 


Supplemental Reading Instruction (no website available) 



Note: Following each program name is the developer’s website address. This table includes programs and practices that were eligible for inclusion in this topic review. 

For more information about the studies reviewed and the WWC methodoiogy, piease see the WWC Enqiish Language Learners Technicai 
Appendices . 
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Appendix A1 Extent of evidence 



Reading achievement Mathematics achievement English language development 



Intervention name 


Number of 
studies 


Sample size 
(schools/ 
students) 


Extent of 
evidence 


Number of 
studies 


Sample size 
(schools/ 
students) 


Extent of 
evidence 


Number of 
studies 


Sample size 
(schools/ 
students) 


Extent of 
evidence 


Arthur 


0 


na 


na 


0 


na 


na 


1 


6/102 


Small 


Bilingual Cooperative 
Integrated Reading 
and Composition 


1 


7/85 


Small 


0 


na 


na 


1 


7/85 


Small 


Enhanced Proactive Reading 


2 


8/131 


Small 


0 


na 


na 


2 


8/131 


Small 


Fast ForWord Language 


2 


Over 7/2401 


Small 


0 


na 


na 


2 


Over 7/2401 


Small 


Instructional Conversations 
and Literature Logs 


2 


11/116 


Small 


0 


na 


na 


2 


11/116 


Small 


Peer Tutoring and 
Response Groups 


3 


5/106 


Small 


0 


na 


na 


3 


5/106 


Small 


Peer-Assisted Learning 
Strategies 


1 


nr/119 


Small 


0 


na 


na 


0 


na 


na 


Reading Mastery 


1 


9/17 


Small 


0 


na 


na 


0 


na 


na 


Read Naturally 


1 


5/60 


Small 


0 


na 


na 


0 


na 


na 


Read Well 


1 


5/33 


Small 


0 


na 


na 


1 


5/33 


na 


Success for All 


1 


8/324 


Small 


0 


na 


na 


1 


8/324 


na 


Vocabulary Improvement 


1 


2/142 


Small 


0 


na 


na 


1 


2/142 


Small 



Program 



nr = not reported 
na = not applicable 

Note: A rating of “moderate to large” requires at least two studies and two schools across studies in one domain and a total sample size across studies of at least 350 students or 14 classrooms. 
Otherwise, the rating is “small.” 

1 . The number of schools in one study was not reported. 
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Appendix A2 Sample characteristics by program 



Program name 


Targeted students (grade levels/ages) 


Students in studies reviewed (grade levels) 


Arthur 


Ages 4-8 


K 


Bilingual Cooperative Integrated Reading and Composition 


Grades 2-5 


2-3 


Enhanced Proactive Reading 


Grade 1 


1 


Fast ForWord Language 


Grades K-12 


K-6 


Instructional Conversations and Literature Logs 


nr 


2-5 


Peer Tutoring and Response Groups 


nr 


1-6 


Peer-Assisted Learning Strategies 


Grades K-12 


3-6 


Reading Mastery 


Grades K-6 


K-4 


Read Naturally 


Grades 1-8 


2-5 


Read Well 


K-1 


2-5 


Success for All 


Pre-K-Grade 8 


K-1 


Vocabulary Improvement Program 


Grades 4-6 


5 



nr = not reported 
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Appendix A3 Summary of statistically significant^ or substantively important^ positive findings^ 





Reading achievement^ 


English language development^ 




Statistically significant 
positive findings 


Reading achievement 
across outcomes 


Statistically significant 
positive findings 


English language development 
across outcomes 


Arthur 










Uchikoshi, 2005 (randomized controlled trial) 


na 


na 


ns 


ns. 

Substantively important 


Bilingual Cooperative Integrated Reading and Composition 

Calderon, Hertz-Lazarowitz, & Slavin, 

1998 (quasi-experimental design) 


ns 


ns. 

Substantively important 


ns 


ns. 

Substantively important 


Enhanced Proactive Reading 

Vaughn, Cirino, et al., 2006 (randomized controlled trial) 


ns 


ns. 

Substantively important 


ns 


ns, 

nsi 


Vaughn, Mathes, etal., 2006 (randomized controlled 
trial with failed random assignment) 


Word attack; 
Passage comprehension 


Statistically significant. 
Substantively important 


ns 


ns, 

nsi 


Fast ForWord Language 










Scientific Learning Corporation, 2004 
(randomized controlled trial) 


na 


na 


Test of Auditory Comprehension 
of Language-Revised 


Statistically significant. 
Substantively important 


Troia, 2004 (quasi-experimental design) 


ns 


ns, 

nsi 


na 


na 


Instructional Conversations and Literature Logs 

Saunders, 1999 (quasi-experimental design) 


Performance assessmenf 


Statistically significant. 
Substantively important 


ns 


ns. 

Substantively important 


Saunders & Goldenberg, 1999 
(randomized controlled trial with a confound) 


Facfual comprehension; 
Interprefive comprehension 


Statistically significant. 
Substantively important 


na 


na 



na = not studied 

ns = not statistically significant 

nsi = not substantively important 



(continued) 



1 . According to WWC criteria, if a program finds a stafisfically significanf effect, there is less than a 5% chance that this difference is due to chance. The level of statistical significance was calculated by the WWC and, where necessary, 
corrects for clustering within classrooms or schools, and for multiple comparisons. For an explanation about the clustering correction, see the WWC Tutorial on Mismatch . For the formulas the WWC used to calculate the statistical 
significance, see the Technical Details of WWC-Conducted Computations . 

2. For rating purposes, the WWC considered the statistical significance of the findings and the magnitude of the effect, also called the effect size. An average effect size is the sum of all the effect sizes of the student outcomes in a study in 
a single domain divided by the number of those outcomes. The WWC considers an average effect size across all student outcomes in one study in a given domain to be substantively important if it is equal to or greater than 0.25. 

3. In this topic review, no studies that met WWC evidence standards with or without reservations addressed mathematics achievement. 

4. No studies showed statistically significant or substantively important negative findings in the domain. For a detailed description of the outcome measures, see Appendix A2 in the WWC intervention report at www.whatworks.ed.aov . 
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Appendix A3 Summary of statistically significant^ or substantively important^ positive findings^ (continued) 





Reading achievement^ 


English language development^ 


Statistically significant 
positive findings 


Reading achievement 
across outcomes 


Statistically significant 
positive findings 


English language development 
across outcomes 


Peer Tutoring and Response Groups 

Jun-Aug, 1985 (randomized controlled trial) 


na 


na 


Language behavior - talking 
to peers; Language behavior - 
addressed from subject to peer 


Statistically significant. 
Substantively important 


Prater & Bermudez, 1983 (randomized controlled trial) 


na 


na 


Total idea units written 


ns. 

Substantively important 


Serrano, 1987 (randomized controlled trial) 


na 


na 


ns 


ns, 

nsi 


Peer-Assisted Learning Strategies 










Saenz, Fuchs, & Fuchs, 2005 (randomized controlled trial) 


ns 


ns, 


na 


na 






Substantively important 






Read Naturally 










Denton, Anthony, Parker, & Flasbrouck, 2004 


ns 


ns, 


na 


na 


(quasi-experimental design) 




nsi 






Read Well 










Denton, Anthony, Parker, & Flasbrouck, 2004 


ns 


ns. 


na 


na 


(randomized controlled trial with differential attrition) 




Substantively important 






Reading Mastery 










Gunn, Biglan, Smolkowski, &Ari, 2000 


ns 


ns. 


na 


na 


(randomized controlled trial) 




Substantively important 






Success for All 










Chambers et al., 2004 (quasi-experimental design) 


ns 


ns. 


na 


na 






Substantively important 






Vocabulary Improvement Program 










Carlo et al., 2004 (randomized controlled 


ns 


ns. 


Word mastery 


ns. 


trial with differential attrition) 




Substantively important 




Substantively important 



na = not applicable 

ns = not statistically significant 

nsi = not substantiveiy important 
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Appendix A4 
Methodology 



WWC Topic Report 



Seventy-three studies, which provided data on 32 Engiish 
ianguage learning interventions, were ciassified for the strength 
of their design. To be fuiiy reviewed, a study had to be a random- 
ized controlled trial or a quasi-experimental design with evidence 
of equating between treatment and comparison groups. Twenty 
studies could not be categorized by an intervention, and are not 
cited in the report. 

Eligibility for review 

Quasi-experiments eiigibie for review include those equating 
through matching or statistical adjustment, regression disconti- 
nuity designs, and singie case designs. Aithough singie case 
designs were identified for the Engiish ianguage iearning review, 
they were not included in intervention reports as we are currently 
developing evidence standards for regression discontinuity 
designs and singie case designs. 

The review considered the properties of measurement instru- 
ments, the percentage of students, ciassrooms, or schoois in the 
study sampie that were not inciuded in the reported resuits, and 
any sample characteristics or events that might serve as aiterna- 
tive explanations for the observed effect. For detaiis please 
see the WWC Evidence Standards . Long-term outcomes were 
preferred over immediate outcomes for inclusion in our anaiysis 
of program effects. 

The research evidence for programs and practices that have 
at ieast one study meeting WWC evidence standards with 
or without reservations is summarized in individual interven- 
tion reports posted on the WWC website. See http://www. 
whatworks.ed.gov . So far, 16 unique or independent studies of 
12 English language learning interventions have met evidence 
standards with or without reservations. The lack of evidence 
for the remaining programs and practices does not mean that 
those programs and practices are ineffective; some programs 
and practices have not yet been studied using a study design 
that permits the WWC to draw any conclusions about their 
effectiveness. And for some studies, not enough data were 
reported (such as descriptive statistics of the findings) to 



English Language Learners 



enable us to confirm statistical findings, so they were not fully 
reviewed. 

Rating of effectiveness 

Each English language learning program that had at least one 
study meeting WWC standards received a rating of effectiveness 
in at least one outcome domain; the rating of effectiveness aims 
to characterize the existing evidence base in a given domain. 

The intervention’s effect based on the research evidence can 
be rated as positive, potentially positive, mixed, no discernible 
effects, potentially negative, or negative. 

The rating of effectiveness takes into account four factors: the 
quality of the research design, the statistical significance of the 
findings, the size of the difference between participants in the 
intervention and the comparison conditions, and the consistency 
in findings across studies (see the WWC Intervention Rating 
Scheme) . 

The level of statistical significance was reported by the study 
authors or, where necessary, calculated by the WWC to correct 
for clustering within classrooms or schools and for multiple com- 
parisons. Because of these corrections, the level of statistical 
significance as calculated by the WWC may differ from the one 
originally reported by the study authors. For an explanation, see 
the WWC Tutorial on Mismatch . For the formulas that we used to 
calculate statistical significance, see Technical Details of WWC- 
Conducted Computations . If the average effect size across all 
outcomes in one study in a single domain is at least 0.25, it is 
considered substantively important, contributing toward the rat- 
ing of effectiveness. See the technical appendices of the English 
language learning intervention report for further details. 

Extent of evidence 

The WWC categorizes the extent of evidence in each domain as 
small or moderate to large (see the What Works Clearinghouse 
Extent of Evidence Categorization SchemeL The extent of 
evidence takes into account the number of studies and the 
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Appendix A4 
Methodology 

(continued) 
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total sample size across the studies that met WWC evidence 
standards with or without reservations.'' 

Improvement index 

The WWC computed an improvement index for each individual 
finding. In addition, within each outcome domain, the WWC 
computes an average improvement index for each domain and 
each study as well as a domain average improvement index 
across studies of the same intervention (see the Technical 



Details of WWC-Conducted Computations^ . The improvement 
index represents the difference between the percentile rank of 
the average student in the intervention condition and the percen- 
tile rank of the average student in the comparison condition. The 
improvement index can take on values between -50 and +50, 
with positive numbers denoting results favorable to the interven- 
tion group. Unlike the rating of effectiveness, the improvement 
index is based only on the size of the difference between the 
intervention and the comparison conditions. 



1. The Extent of Evidence Categorization was developed to tell readers how much evidence was used to determine the intervention rating, focusing on the 
number and size of studies. Additional factors associated with a related concept, external validity— such as the students’ demographics and the types 
of settings in which studies took place— are not taken into account for the categorization. 
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tion from the effects of the school. Complete data were not reported in one study (El Vista), so the WWC could not calculate effect sizes. 
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